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ABSTRACT 

This study is primarily focused on assessing the students' logical thinking and cognitive 
levels in an online collaborative environment. The aim is to investigate whether the online 
collaboration has significant impact to the students' cognitive abilities. The assessment of 
the logical thinking involved the use of the online Group Assessment Logical Thinking (GALT) 
test that has been conducted in two phases; before and after the online collaborative 
activities. 

The sample of respondents for this study is sixty first year Diploma in Computer Science 
students from Universiti Teknologi MARA (UiTM) Perlis, Malaysia where they were divided 
into fifteen collaborative groups. These collaborative groups were then engaged in a 3-hour 
session of collaborative activities via the Online Collaborative Learning System (OCLS). The 
results for this study has revealed that the online collaborative learning has significant 
impact to the students' logical thinking levels with the increment of 21.7% high logical 
thinkers with p-value<0.05 (sig. 2-tailed). Meanwhile, the investigation of the students' 
cognitive levels is being done by monitoring the students' abilities to solve the given 
questions via OCLS. The questions have been previously constructed according to the 
Bloom's taxonomy cognitive domain. 

The results have also revealed that the students at the early stage of learning programming 
are able to solve complex programming problems at the cognitive level Application and 
Analysis. There was also a strong correlation between students' logical thinking skills with 
their abilities to solve problems in an online platform with r= 0.631, significant at 0.012. 
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INTRODUCTION 

The usage of information technology and communications these days have radically 
transformed the face of our education system especially in the area of flexible and distance 
learning. The mass used of online learning platforms is one of the evidences that show the 
rapid expansion of the Internet. 
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According to Tsai, Wu, Elston and Chen (2011), in this day and age, the Internet is not just 
purposeful for searching and surfing, but also has expanded to support varieties of 
educational activities including communication and online collaboration. Owen, Grant, Sayers 
and Facer (2006) have also acknowledged that the trends of e-learning styles have positively 
escalating towards the c-learning styles that emphasize the combinations of community, 
communicative and collaborative learning. 

Collaborative learning has been famously implemented in the teaching and learning 
environment. Thus, the effectiveness of incorporating collaborative learning in physical 
classes has been proven and well-documented (Mahfudzah, Fazlin Marini, Khairunnisa, 
2010). Although there are many ways to describe collaborative learning, the main purpose 
of implementing group collaboration is to bring the team together in order to accomplish a 
common goal (Mahfudzah, Muhaini, Fazlin Marini, 2013). This has also becomes the main 
purpose of many online collaborative platforms. Varieties of Web 2.0 tools have been used to 
facilitate communication, collaboration activities and problem-solving process over 
geographic distances and from dispersed locations. 

In teaching and learning computer programming, there are numbers of online collaborative 
learning platforms that have been developed to support online collaboration. Codewitz, for 
instance, is a project that used web-based platform to support learning programming 
concepts using visualizations (Lahtinen, Ala-Mutka & Jarvinen, 2005). Meanwhile, Van Hiele 
Web-Based Learning System that was developed by Chen and Chih (2006) has also 
incorporated a range of Web 2.0 tools such as electronic mails, discussion board, assignment 
and tutorial modules and knowledge management. 

Other than that, Cavus (2007) have also developed a Learning Management System using 
Moodle LMS with the combination of collaborative learning tool called GREWPtool to support 
a virtual teaching environment for learning programming languages. Current researches 
involve more complex and highly interactive systems such as the Online Collaborative 
Learning System (OCLS) designed by Mahfudzah, et. al (2013), Programming Assignment 
assessment System (PASS) by Law, Lee, and Yu (2010), Supporting Collaboration and 
Adaptation in a Learning Environment (SCALE) by Verginis, Gogoulou, Gouli, Boubouka and 
Grigoriadou (2011) and AutoLEP by Wang, Su, Ma, Wang and Wang (2011). 

Many of these online collaborative platforms are being developed not only to support the 
teaching and learning of computer programming, but also to enhance the students' 
performance, improve their learning styles as well as their problem-solving skills. For 
example, by using SCALE, the students are expected to actively participate with their own 
progress in programming by referring to the information and feedbacks given by the system. 
Through this, students will become increasingly aware of their own performance in 
programming courses (Verginis, et.al, 2011). 

Meanwhile, by using PASS, the well facilitated e-learning environment in this system is 
claimed to be advantageous in motivating students' learning process and activities (Law, 
et.al, 2010). Besides that, the used of OCLS has been proven to give positive impact towards 
students' performance where the t-test analysis had showed significant value of 0.01, which 
is less than 0.05 (sig. 2-tailed) (Mahfudzah, et.al, 2013). Previously, AutoLEP, which is 
specifically designed to assist novice programmers in mastering their programming skills, 
has been proven effective in improving students' learning experience hence effectively ease 
the workload of the instructors (Wang, et.al, 2011). Other research claimed that by utilizing 
the collaborative e-learning systems in teaching and learning programming, it has positively 
improved the interactions and participation among the students and the tutors as well as 
successfully facilitated real-time compilation of activities for the students (Cheung, 2006). 
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While there are a lot of online collaborative systems developed over the years to support the 
teaching and learning of computer programming courses, whilst the implementation has also 
been proven effective, however, there is still lack of study that focus on how the online 
collaborative system can effectively influences the students' cognitive abilities in 
programming. Previous studies have emphasized that cognitive abilities are crucial for the 
students to master in programming. White and Sivitanides (2002) claimed that cognitive 
abilities are one of the factors that determine the success in computer programming courses. 
Among the cognitive abilities needed to become an expert programmer as highlighted by 
Mohd Nasir, Nor Azilah and Irfan Naufal (2006) are analytic processing, problem-solving 
skills, reasoning and logical thinking skills as well as mathematical skills. 

Mahfudzah, et. al. (2010) have also claimed that logical thinking skills are one of the 
important factors that determine the students' abilities to analyze, plan and solve problems 
in programming. According to Piagetian theory, there are four stages of cognitive 
development comprises of sensory-motor, pre-operational, concrete operational and formal 
operational where the ability to think logically is determined in the period of abstract process 
and has been claimed as the higher cognitive skill (Atherton, 2005). Learners will use their 
logical thinking skills to solve problems by engaging in varieties of mental practices or by 
doing some abstraction and generalization (Yaman, 2005). 

In order to measure the level of logical thinking skills, the Group Assessment Logical 
Thinking (GALT) test can be utilized as the measuring tool (Tuna, Biber & incikapi, 2013; Lay, 
2009; McConnell, Steer, Owens & Knight, 2005; Yaman, 2005). The GALT test was developed 
by Roadrangka, Yeany and Padila (1983) and they have identified six logical subscales which 
are conservational reasoning, proportional reasoning, controlling variables, probabilistic 
reasoning, correlational reasoning and combinatorial reasoning. Therefore, the lack of study 
that focus on the assessment of the students' cognitive abilities especially in their logical 
thinking and cognitive skills in an online collaborative environment has raised the research 
questions for this study; i) how can we measure the students' logical thinking and cognitive 
levels?, ii) do the online collaborative activities significantly affect the students' logical 
thinking levels?, iii) do the students' logical thinking skills have any correlation with their 
abilities to solve questions collaboratively in an online platform? Therefore, this study is 
aimed to investigate the students' logical thinking levels before and after they engage in the 
online collaborative activities. This is to assess whether the online collaboration has 
significant impact on their logical thinking skills or otherwise. Apart from that, this study will 
also measure the students' cognitive levels based from the questions posted online. Each 
question represents different cognitive levels where the collaborative groups will be asked to 
solve the questions collaboratively in an online collaborative environment. In this study, the 
Online Collaborative Learning System (OCLS) will be used to facilitate the online 
collaboration. 

MATERIALS & METHODS 

The research method for this study is divided into four main phases as mentioned below: 
Research Populations and Sample 

The population of this study was the students enrolled in Diploma in Computer Science from 
Universiti Teknologi MARA (UiTM) Perlis, Malaysia. The sample of the respondents was sixty 
male and female students from the first year of the particular program. Prior to this, 
fundamentals programming course has been taught to these sample groups for the period of 
four months, meaning that, they have already had prior knowledge in basic programming 
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concepts. Before this research was conducted, collaborative activities have not been 
implemented either in the classrooms or via online platforms. 

Pre-Logical Thinking Test 

A week before they attended the online collaborative session, the sample groups were asked 
to answer an online logical thinking test. The online logical thinking test used in this 
research was an electronic version of a pen and paper based GALT test that has been widely 
used to measure logical thinking levels in the field of education. The Cronbach's alpha 
reliability coefficient of the logical thinking test was recorded at 0.52 which is considered 
moderate for use in the study (Lay, 2009). The GALT test used in this study consists of twelve 
items and ranging from six subscale measures for logical operations as highlighted by 
Roadrangka, et.al (1983). The subscale measurement as stated in the Table 1 below 
reflected the Piagetian cognitive reasoning abilities modes. 

Table: 1 

The six subscale measures for logical operations in GALT test 


Subscales 

Item No. 

Item Descriptor 

Conservational reasoning 

1 

Piece of Clay 

2 

Metal Weigh 

Proportional reasoning 

3 

Glass Size 

4 

Scale 

Controlling variables 

5 

Pendulum Length 

6 

Ball 

Probabilistic reasoning 

7 

Square and Diamonds #1 

8 

Square and Diamonds #2 

Correlational reasoning 

9 

The Mice 

10 

The Fish 

Combinatorial reasoning 

11 

The Dance 

12 

The Shopping Center 


The pre-logical thinking test results were then assessed where the students who were able 
to answer more than six items correctly were categorized as high logical thinkers (HLT), 
while the others with scores lower than six were categorized as low logical thinkers (LLT). 
These scores were recorded and used to divide the students into fifteen collaborative groups. 
Each collaborative group consists of a mixture of HLT and LLT students. 

Online Collaborative Activities 

The collaborative groups were then asked to engage in the collaborative activities in the 
online collaborative system. As mentioned before, in this study, the Online Collaborative 
Learning System (OCLS) will be used to facilitate the online collaboration. Figure 1 below 
depicts some of the user interfaces of OCLS where the collaborative groups used to engage in 
the online collaborative activities. The details of the system's development and architecture 
have been discussed by Mahfudzah et.al. (2013). 
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Figure: 1 

OCLS's user interface that support online collaborative 
activities Mahfudzah et.al. (2013) 


Via OCLS, the questions were posted by the lecturers and the collaborative groups were 
asked to provide the solutions in a 3-hours online collaborative session. The questions given 
to the collaborative groups were developed by the lecturers with more than seven years 
teaching experience in programming. The questions were constructed based from the 
Fundamentals of Computer Problem-Solving course which comprises six sub topics with 
different cognitive levels. 

In this case, the Bloom's taxonomy cognitive domain which has six stages; knowledge (Cl), 
comprehension (C2), application (C3), analysis (C4), synthesis (C5) and evaluation (C6) was 
used as the guideline in the construction of the questions. 

Table: 2 lists the distribution of questions that were used in this study. 
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Table: 2 

The distribution of questions constructed according 
to the Bloom's taxonomy cognitive domain 


Question 

No 

Name Of 
Question 

Topic Covered 

Chapter 

Cognitive 

Level 

Marks 

Q1 

Working Hours 

Sequential 

2 

C3-Application 

6 

Q2 

Calculate the 
Income 

Selection 

3 

C4 - Analysis 

17 

Q3 

Prime Number 

Repetition 

4 

C4 - Analysis 

8 

Q4 

Rectangle 

Repetition 

4 

C4 - Analysis 

5 

Q5 

Diamond Shape 

Repetition 

4 

C5 - Synthesis 

10 

Q6 

Pantun 

Selection 

3 

C6 - Evaluation 

20 

Q7 

The Tallest 
Student 

Functions 

5 

C4 - Analysis 

10 

Q8 

Odd Numbers 

Arrays 

6 

C4 - Analysis 

12 


According to Table 2, eight questions have been given to the collaborative groups ranging 
from the application (C3) stage until the higher order thinking skills which were synthesis 
(C5) and evaluation (C6). 

This was because the lecturers were taking into accounts the students' previous learning 
experience in the fundamentals of computer programming paper. The online collaborative 
session was done in a controlled 3-hour session with the lecturers giving feedbacks and 
results promptly. 

This was to encourage the collaborative groups to engage in a more interactive, 
communicative and collaborative online learning environment. 

Post-Logical Thinking Test 

After the 3-hour session has ended, the students were again asked to take the online GALT 
test in order to investigate the improvements to the students' logical thinking levels after the 
online collaborative activities. 

The post-logical thinking test results were also assessed and recorded. Later, both the pre¬ 
test and post-test logical thinking scores were being analyzed using IBM SPSS Statistics 
19.0. 


Types of analysis used in this study are descriptive analysis and paired-samples t-test to find 
the significant impact of online collaboration towards students' logical thinking abilities and 
Spearman Rho's Bivariate correlations to find the correlations between the students' logical 
thinking levels and their abilities to solve programming questions in an online platform. 


89 
















RESULTS & DISCUSSIONS 


Students' Logical Thinking Assessments 

As mentioned in the previous section, the sample groups were asked to answer the pre and 
post-logical thinking test using the online GALT test. 

The pre-logical thinking test was taken a week before the sample groups engage in the 
online collaborative activities. Table: 3 below depict the results of pre-logical thinking tests 
for all sixty students. 


Table: 3 

Pre-logical thinking test's results for all students 



Frequency 

Percent 

Valid Percent 

Cumulative Percen 
t 

Question 

1 

1 

1.7 

1.7 

1.7 

2 

6 

10.0 

10.0 

11.7 

3 

18 

30.0 

30.0 

41.7 

4 

5 

8.3 

8.3 

50.0 

5 

12 

20.0 

20.0 

70.0 

6 

11 

18.3 

18.3 

88.3 

7 

5 

8.3 

8.3 

96.7 

8 

1 

1.7 

1.7 

98.3 

9 

1 

1.7 

1.7 

100.0 

Total 

60 

100.0 

100.0 



From the results in Table 3, it shows that about 42 students (70%) have scored 5 marks and 
below, which made them low level thinkers (LLT). Meanwhile, only 18 of the students (30%) 
have scored more than 6 marks, which identified them as high logical thinkers (HLT). 

Moreover, there was only one student who was able to answer up to nine questions correctly 
and none of the students have scored a full twelve marks in the pre-logical thinking test. 

The mean score for this test is 4.42 which described that the overall score of the logical 
thinking abilities among the students were relatively low. 

These pre-logical thinking marks were then used to divide the students into fifteen small 
collaborative groups where each group consists of four members from the mixture of LLT and 
HLT. 

Table: 4 display the distribution of the collaborative groups (labeled as G1 until G15) with 
their mean logical thinking scores were between 4.25 until 4.50. 
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Table: 4 

The pre-logical thinking's mean scores for each collaborative group 


NO 

GROUP 

Pre-Logical Thinking Mean Score 

1 

G1 

4.25 

2 

G2 

4.50 

3 

G3 

4.50 

4 

G4 

4.25 

5 

G5 

4.50 

6 

G6 

4.25 

7 

G7 

4.50 

8 

G8 

4.50 

9 

G9 

4.25 

10 

G10 

4.50 

11 

Gil 

4.25 

12 

G12 

4.50 

13 

G13 

4.50 

14 

G14 

4.50 

15 

G15 

4.50 


Meanwhile, Table: 5 shows the results for the post-logical thinking test. As explained 
previously, the post-logical thinking test was taken after the collaborative groups engaged in 
a 3-hour session of online collaborative activities via OCLS. 

The aim was to assess whether the students' logical thinking levels have increased after the 
online collaboration session. 

From the analysis revealed in Table 5, the results have showed an increment of HLT where 
the number of students who scored more than 5 marks have increased to 31 students 
(51.7%). Thus, adding 13 more students in the HLT's group with 21.7% increment. 

Meanwhile, the other 29 students have remained as LLT (48.3%). The results have also 
showed that at least one student was able to answer more than ten questions in the post- 
logical thinking test. 

However, none of them were able to answer all twelve questions correctly. The mean score 
for the post-logical thinking test is 5.20 which showed slight increment from the pre-logical 
thinking's mean score. 
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Table: 5 

The post-logical thinking results for all students 



Frequency 

Percent 

Valid Percent 

Cumulative Percent 

Question 

2 

5 

8.3 

8.3 

8.3 

3 

11 

18.3 

18.3 

26.7 

4 

2 

3.3 

3.3 

30.0 

5 

11 

18.3 

18.3 

48.3 

6 

18 

30.0 

30.0 

78.3 

7 

9 

15.0 

15.0 

93.3 

8 

2 

3.3 

3.3 

96.7 

9 

1 

1.7 

1.7 

98.3 

10 

1 

1.7 

1.7 

100.0 

Total 

60 

100.0 

100.0 



Table: 6 below depict the post-logical thinking mean scores for the fifteen collaborative 
groups after the online collaborative activities. The results have exhibited that all fifteen 
collaborative groups have showed an increment in their mean scores between 4.50 until 
6.25. 


Table: 6 

The post-logical thinking's mean scores for each collaborative group 


NO 

GROUP 

Post - Mean Score 

1 

G1 

4.50 

2 

G2 

5.25 

3 

G3 

5.00 

4 

G4 

5.00 

5 

G5 

5.25 

6 

G6 

5.50 

7 

G7 

5.00 

8 

G8 

5.25 

9 

G9 

5.00 

10 

G10 

6.25 

11 

Gil 

4.75 

12 

G12 

5.25 

13 

G13 

5.50 

14 

G14 

5.50 

15 

G15 

5.00 
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For further investigation, a paired-sample t-test was conducted in order to investigate the 
significant impact on students' logical thinking levels after the online collaborative activities. 
Table 7 below reveals that on average, all fifteen collaborative groups have increased their 
logical thinking abilities with mean score of 3.133 and statistic test (t) result is +8.951, with 
the p-value<0.05 (sig. 2-tailed). 


Table 7: The paired-samples t-test result 




Paired Differences 







Std. 

Std. 

Error 

95% Confidence 
Interval 

of the Difference 



Sig. (2- 
tailed) 



Mean 

Deviation 

Mean 

Lower 

Upper 

t 

df 

Pair 1 

Post mean 
score-Pre mean 

score 

3.133 

1.356 

.350 

2.383 

3.884 

8.951 

14 

.000 


Therefore, based from all of these analyses, it has been proven that the online collaborative 
activities designed and implemented for this study specifically for teaching and learning 
programming, have significant impact on the students' logical thinking levels. 

Students' Cognitive Levels Based from the Questions They have Solved 

Table: 8 below display the distribution of questions that the collaborative groups have 
answered via OCLS. Previously, the questions developed for this study have been constructed 
based on the Bloom's taxonomy cognitive domain. 


Table: 8 

The questions that have been answered by the collaborative groups 



Q1 (C3) 

Q2 (C4) 

Q3 (C4) 

Q4 (C4) 

Q5 (C5) 

Q6 (C6 

Q7 (C4) 

Q8 (C4) 

Mean 

1.00 

.53 

.20 

.20 

.13 

.00 

.53 

.20 

Std. Deviation 

.000 

.516 

.414 

.414 

.352 

.000 

.516 

.414 

Variance 

.000 

.267 

.171 

.171 

.124 

.000 

.267 

.171 

Sum 

15 

8 

3 

3 

2 

0 

8 

3 


By referring to Table 8, it shows that all fifteen collaborative groups were able to solved 
question Q1 that covers the topic sequential control structure with cognitive level C3 
(application). Meanwhile, eight collaborative groups were able to answer question Q2 
(selection control structure) and Q7 (functions) successfully, with both questions 
constructed for cognitive level C4 (analysis). However, only three groups were able to solved 
questions Q3, Q4 and Q8 where each question represents topics from repetition control 
structure and arrays. All three questions are designed to conform to level C4 (analysis) in 
Bloom's taxonomy cognitive domain. From the findings of this study, it has also revealed 
that questions with higher order thinking skills which are question Q5 and Q6 with cognitive 
level C5 (synthesis) and C6 (evaluation) were considered hard for the students to solve 
where none of the collaborative groups were able to successfully solved question Q6 even 
though the question was also constructed under the topic selection control structure. 
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Correlations between Collaborative Groups' Logical Thinking Mean Scores 
with Their Abilities to Collaboratively Solve Questions in an Online Environment 
In-depth investigation has also revealed the significant correlation between the 
collaborative groups' logical thinking mean scores with their abilities to solve the given 
questions in an online environment. Table 9 below exhibits the Spearman's Rho Bivariate 
correlations analysis that shows strong correlation with r= 0.631, at the rate of 39.8% and 
significant value is 0.012, which is less than 0.05 (sig. 2-tailed). This analysis has proved that 
the logical thinking ability is one of the important factors that determine the students' 
success rates in solving computer programming questions in an online collaborative 
environment. 


Table: 9 

Correlations between collaborative groups' logical 
thinking mean scores with their abilities to solve questions in OCLS 



Collaborative groups' 
mean score 

Ability to solve 
questions 

Spearman's rho 

Collaborative 
groups' mean 
score 

Correlation 

Coefficient 

1.000 

.631** 

Sig. (2-tailed) 

. 

.012 

N 

15 

15 

Ability 
to solve 
questions 

Correlation 

Coefficient 

.631** 

1.000 

Sig. (2-tailed) 

.012 

. 

N 

15 

15 


CONCLUSION 

Online collaborative system for teaching and learning programming has been widely 
developed and implemented; hence provide many benefits to the students and the lecturers 
as well. Among the advantages that have been claimed by many researchers are the online 
collaboration is able to improve students' interaction and participation in the online 
environment, aids novice programmers in improving their programming skills and also 
improve students' learning styles and performance. 

In this study, a series of assessments have been conducted in order to investigate whether 
the online collaboration for teaching and learning programming course have positively affect 
the students' cognitive abilities or otherwise. From the findings of this study, it has revealed 
that with proper cognitive assessments, group's formulation and construction of questions; 
the online collaborative activities seemed to give significant impact to the students' cognitive 
abilities especially in their logical thinking skills. 

Besides that, through this study, the researchers have realized that the students at their 
early stage of learning programming were able to solve programming questions with 
cognitive levels as high as application and analysis; taking into considerations few factors 
such as the questions are carefully constructed based from the Bloom's taxonomy cognitive 
domain by experienced lecturers and the students' previous knowledge in fundamentals 
programming concepts. 

Although the results have showed positive effects towards students' cognitive abilities, 
further investigation need to be conducted by looking into other factors that could contribute 
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to the success of online collaboration for teaching and learning programming such as 
providing the online collaborative system with learner models for personal reflections, online 
feedbacks for students' performance or other types of assessments such as online peer and 
expert evaluations. 
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